Skip to content

Add harbor_env — serve Harbor task directories as an OpenEnv environment - #1018

Open
thegovind wants to merge 9 commits into
huggingface:mainfrom
thegovind:harbor-env
Open

Add harbor_env — serve Harbor task directories as an OpenEnv environment#1018
thegovind wants to merge 9 commits into
huggingface:mainfrom
thegovind:harbor-env

Conversation

@thegovind

@thegovind thegovind commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

A new environment, envs/harbor_env, that runs any Harbor task directory as a Gymnasium-style OpenEnv environment.

The task directory is the interface. A directory that harbor run accepts is served here unchanged — no conversion step, no re-authoring, no second copy of the data, and no producer-specific code in OpenEnv. That covers Terminal-Bench-lineage tasks and everything Repo2RLEnv generates from a GitHub repository, so a task set built for evaluation is also a training environment.

# serve the bundled example task
uv run --project envs/harbor_env server

# or point it at your own task set — a directory or a Hub dataset
HARBOR_TASKS_DIR=./tasks HARBOR_MODE=docker uv run --project envs/harbor_env server
HARBOR_TASKS_DIR=hf://datasets/my-org/click-tasks uv run --project envs/harbor_env server

Where this sits

Harbor and OpenEnv are not competitors — they sit at different layers, and they agree on the one contract that matters: the reward is produced inside the environment and only forwarded.

flowchart LR
    P["Task producers<br/>Repo2RLEnv · Terminal-Bench<br/>· hand-written"]
    T["Harbor task directory<br/><i>the format</i>"]
    H["harbor run<br/><i>batch evaluation</i>"]
    E["envs/harbor_env<br/><b>NEW</b> — episode loop"]
    TR["Trainer / RL loop<br/>reset · step · state"]

    P -->|emit| T
    T --> H
    T -->|"served unchanged"| E
    E --> TR

    style E fill:#c8e6c9,stroke:#2e7d32,stroke-width:3px
    style T fill:#e3f2fd,stroke:#1565c0
Loading
Harbor OpenEnv
Kind of thing A file format + a batch runner A serving standard + runtime
Unit A task directory on disk A running service
Interaction One-shot: drop agent in sandbox, verify at the end Stepwise episode loop, resettable by the trainer
Reward tests/test.sh/logs/verifier Observation.reward, produced inside the env

harbor_env is the bridge, and it generalizes the existing envs/tbench2_env/ pattern. The one real behavioural difference: tbench2 derives pass/fail from the verifier's exit code, while Harbor rewards are graded floats, so harbor_env reads the value the verifier wrote.


An episode

sequenceDiagram
    participant TR as Trainer <br/>(infrastructure)
    participant E as HarborEnvironment
    participant S as Sandbox <br/>(local or docker)
    participant V as tests/test.sh

    TR->>E: reset(task_id="fix-sum-bug")
    E->>S: boot backend, seed working directory
    E-->>TR: Observation(instruction=instruction.md)

    loop agent works
        TR->>E: step(exec / read / write)
        E->>S: run in the working directory
        S-->>E: output
    end

    TR->>E: step(evaluate)
    E->>S: clear /logs/verifier, stage tests/ -> /tests
    E->>V: bash /tests/test.sh
    V-->>S: reward.json / reward.txt
    E-->>TR: Observation(reward=0.83, done=True)

    Note over E,V: reward.json first, then reward.txt —<br/>exactly Harbor's precedence
    Note over TR,E: evaluate + solve are infrastructure-only,<br/>never in the agent's action space
Loading

Invariants held

  • Agents cannot reset. reset / step / state stay on the infrastructure side. evaluate and solve are orchestration controls, not agent actions — an agent that could grade on demand could end its own episode, and one that could solve could hand itself the answer.
  • Rewards live inside the environment. harbor_env forwards what tests/test.sh wrote and never computes a reward. If the verifier wrote no reward file, the result is reward=None plus an explicit error — not a 0.0. A fabricated zero is indistinguishable from a genuine failure and would poison a training run.
  • Client/server separation. client.py imports nothing from server/.
  • The agent cannot reach the answer. read and write go through resolve_within(), confined to the working directory, so /tests and /solution are unreachable. /logs/verifier is wiped immediately before the verifier runs, so a reward file planted during exec cannot survive into the score.

Two execution modes

flowchart TD
    A["reset(task_id)"] --> B{"Where is the task's<br/>starting state?"}
    B -->|"ships environment/ seed files<br/><i>self-contained</i>"| C["local mode OK<br/><i>subprocesses, no Docker —<br/>works on HF Spaces</i>"]
    B -->|"Dockerfile / compose /<br/>docker_image"| D{"HARBOR_MODE"}
    D -->|docker| E["run inside the task's<br/>own image"]
    D -->|local| F["refuse with an actionable message<br/><i>rather than grading an empty directory</i>"]

    style C fill:#e8f5e9,stroke:#2e7d32
    style E fill:#e8f5e9,stroke:#2e7d32
    style F fill:#ffebee,stroke:#c62828
Loading
task 'pallets__click-2951' keeps its starting state in a container image
(environment/Dockerfile), which the local backend cannot reproduce.
Run the server with HARBOR_MODE=docker, or use a task that ships its files in environment/.

local mode is documented as a filesystem boundary, not a security boundaryexec runs as the server's own user with the server's own environment. Serve task sets you trust there; use docker for anything else.


Verification

The interop claim is the whole point of this PR, so it was measured rather than asserted.

The bundled examples/tasks/fix-sum-bug task is written to run under every runtime (it prefers $HARBOR_* variables and falls back to Harbor's absolute paths). It grades identically in all three:

Runtime Untouched task After solution/solve.sh
harbor run -a nop / -a oracle (harbor 0.20.0) 0.600 1.000
harbor_env local mode 0.600 1.000
harbor_env docker mode 0.600 1.000

Reproduce row one with harbor run -p envs/harbor_env/examples/tasks/fix-sum-bug -a nop, and the others with examples/validate_taskset.py.

Separately, a real Repo2RLEnv-emitted pr_runtime task — produced by repo2rlenv's own emitter, so version = "1.0" rather than Harbor's schema_version, plus [metadata.repo2env], WORKDIR /workspace, and a tests/test.sh writing /logs/verifier/reward.txt — scored 0.0 for a no-op agent and 1.0 for the oracle under both harbor run and harbor_env docker mode, with reward-details.json surfacing intact as observation.info["reward_details"].


Note

High Risk
New environment runs attacker-controlled shell in local or Docker sandboxes and forwards training rewards; mis-grading or weak isolation could affect RL data and host security.

Overview
Introduces envs/harbor_env, a new OpenEnv environment that serves Harbor task directories unchanged (including Repo2RLEnv output) over the standard reset/step API, with a FastAPI/WebSocket server, HarborEnv client, and wire types for exec / read / write plus infrastructure-only evaluate and solve.

Execution is split between docker (task images, policy enforcement) and local (subprocesses under a per-episode tree for Spaces); local mode refuses tasks that need images or policies it cannot enforce. Rewards are read from verifier artifacts (reward.json then reward.txt, with reward-details.json in info); missing files yield reward=None, not 0.0. Sandboxing limits agent I/O to the workdir, stages /tests only at verify time, and clears verifier logs before grading.

Also adds a bundled fix-sum-bug example task, quickstart / validate_taskset scripts, Docker image defaults, and docs entries (environments/harbor, catalog card, toctree).

Reviewed by Cursor Bugbot for commit 5079a77. Bugbot is set up for automated code reviews on this repo. Configure here.

…Env env

Adds envs/harbor_env, which runs any Harbor task directory
(https://www.harborframework.com/docs/tasks) as a Gymnasium-style OpenEnv
environment. The task directory is the interface: a directory that
`harbor run` accepts is served unchanged — no conversion step, no second copy
of the data, and no producer-specific code in OpenEnv. That covers
Terminal-Bench-lineage tasks and everything Repo2RLEnv generates from a GitHub
repository, so a task set built for evaluation is also a training environment.

Design
------
The environment is deliberately thin and split by concern:

  task.py     parses task.toml and discovers task directories
  sandbox.py  Harbor's sandbox contract; local and docker backends
  reward.py   Harbor's reward-file contract
  harbor_env_environment.py  reset/step/state over the three above
  app.py      FastAPI

Rewards are always produced by the task's own tests/test.sh and only forwarded,
per OpenEnv's rewards-in-environment invariant. reward.json is read first and
reward.txt is the fallback, matching Harbor. A verifier that writes no reward
file yields reward=None and an explicit error rather than a fabricated 0.0,
which would be indistinguishable from a genuine failure and would poison
training data.

exec/read/write are the agent's action space; evaluate and solve are
orchestration controls and are never exposed to the agent, keeping the dual API
boundary intact. read and write are confined to the working directory so an
agent cannot reach /tests or /solution, and the verifier log directory is
cleared immediately before the verifier runs so a planted reward file cannot
survive into the score.

Two backends: `local` runs subprocesses under a per-episode directory tree with
Harbor's paths exported as HARBOR_* variables (no Docker, so it works on HF
Spaces), and `docker` runs inside the task's own image, which is required for
tasks whose starting state lives in an image. Local mode refuses image-backed
tasks with an actionable message instead of grading an empty directory, and is
documented as a filesystem boundary rather than a security one.

Task sets load from a local directory or straight from a Hugging Face dataset
(hf://datasets/<org>/<name>[@rev]), including the tasks/<id>/ layout that
`repo2rlenv push` writes.

Verification
------------
The bundled examples/tasks/fix-sum-bug task is written to run under every
runtime and grades identically on all three — 0.600 untouched, 1.000 after
solution/solve.sh — under `harbor run` (harbor 0.20.0), harbor_env local mode,
and harbor_env docker mode.

A real Repo2RLEnv-emitted pr_runtime task (produced by repo2rlenv's own
emitter: version = "1.0", [metadata.repo2env], environment/Dockerfile with
WORKDIR /workspace, tests/test.sh writing /logs/verifier/reward.txt) scores 0.0
for a no-op agent and 1.0 for the oracle under both `harbor run` and
harbor_env docker mode, with reward-details.json surfaced intact as
observation.info["reward_details"].

tests/envs/test_harbor_env.py adds 50 hermetic tests (no Docker, no network)
covering task parsing including Repo2RLEnv's `version` spelling, catalog
discovery, the reward-file precedence chain, workdir escape rejection, the
agent/orchestration boundary, and the client/server wire contract. Full suite:
1463 passed, 130 skipped.

Also adds examples/quickstart.py, examples/validate_taskset.py (an oracle sweep
to run before training on a generated task set), the docs stub, and the
environment catalog entry.
Copilot AI review requested due to automatic review settings July 28, 2026 14:48
@bot-ci-comment

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Comment thread envs/harbor_env/server/harbor_env_environment.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new self-contained OpenEnv environment, envs/harbor_env, that serves Harbor task directories (including Repo2RLEnv-emitted tasks) as reset/step episodes with verifier-produced rewards forwarded from Harbor’s reward-file contract.

Changes:

  • Introduces harbor_env server/client/models, plus task discovery/parsing, sandbox backends (local + Docker), and reward parsing.
  • Ships a bundled end-to-end example Harbor task (fix-sum-bug) and runnable validation/quickstart scripts.
  • Adds docs and catalog entries for the new environment.

Alignment Review Report

Automated Checks

  • Lint: UNKNOWN — I can’t execute bash .claude/hooks/lint.sh in this review context.
  • Debug code: FOUND — src/ contains existing print(...) statements (e.g. src/openenv/core/mcp_client.py) and existing TODOs; the hook is informational (exits 0) but will report these.

Open RFCs Context

  • In Review: RFC 000, 001, 002, 003, 005
  • Draft: RFC 010
    No direct conflicts identified with the Harbor environment implementation; this PR mostly introduces a new environment rather than changing core contracts.

Tier 1: Fixes Required

  • envs/harbor_env/server/sandbox.py:489-496 — DockerSandbox.upload_dir() builds tar archives with duplicate entries due to tarfile.add() default recursion (stored PR comment ID: 001).
  • envs/harbor_env/server/harbor_env_environment.py:243-251 — _do_read() should reject empty paths with a clear error (stored PR comment ID: 002).
  • envs/harbor_env/server/harbor_env_environment.py:253-261 — _do_write() should reject empty paths with a clear error (stored PR comment ID: 003).

Tier 2: Alignment Discussion

Principle Conflicts

ALIGNMENT FLAG: Orchestration-only controls (evaluate, solve) live in the same action schema as agent actions.

  • Principle/RFC at stake: System invariant “Agent isolation” / “Dual API boundary” (INVARIANTS.md)
  • The concern: The code relies on the trainer/policy wiring to keep evaluate/solve out of the agent’s action space. If a policy is allowed to choose freely among action types, it can end episodes or apply the oracle solution.
  • Suggested reviewer: @darktex

ALIGNMENT FLAG: local mode executes arbitrary shell commands as the server user (no container boundary).

  • Principle/RFC at stake: “Container isolation” (PRINCIPLES.md / INVARIANTS.md)
  • The concern: This is documented as “not a security boundary,” but it’s still a sharp edge worth explicitly validating as acceptable for a new environment (especially for deployment defaults).
  • Suggested reviewer: @darktex
RFC Conflicts

None identified.

Show a summary per file
File Description
tests/envs/test_harbor_env.py Adds hermetic unit + end-to-end tests for task parsing, reward semantics, sandbox confinement, and client/server wire compatibility.
envs/harbor_env/server/task.py Implements Harbor task parsing, schema compatibility (schema_version vs version), discovery via TaskCatalog, and task-source resolution (local or HF dataset).
envs/harbor_env/server/sandbox.py Adds sandbox contract + implementations for local subprocess execution and Docker execution, including task staging and verifier/oracle execution.
envs/harbor_env/server/reward.py Implements Harbor reward-file precedence (reward.json then reward.txt) and optional Repo2RLEnv sidecar surfacing.
envs/harbor_env/server/harbor_env_environment.py Orchestrates reset/step lifecycle over tasks + sandbox, forwarding verifier rewards into observations/state.
envs/harbor_env/server/Dockerfile Builds a runnable server image for harbor_env with uv/venv setup and runtime deps.
envs/harbor_env/server/app.py FastAPI app wiring via openenv server utilities and env factory configuration.
envs/harbor_env/server/init.py Exposes server-side public API surface (Environment, Sandbox backends, parsing helpers).
envs/harbor_env/README.md Environment README/Space metadata, usage, format contract, and operational guidance.
envs/harbor_env/pyproject.toml Defines the env package, deps (incl. docker SDK), and the server script entry point.
envs/harbor_env/openenv.yaml Registers the environment for OpenEnv tooling (name/version/runtime/types).
envs/harbor_env/models.py Defines shared Pydantic wire models (action/observation/state) for client/server separation.
envs/harbor_env/client.py Implements HarborEnv client wrappers and result/state parsing.
envs/harbor_env/init.py Package exports for client + wire types.
envs/harbor_env/examples/validate_taskset.py Provides an oracle-sweep validator to verify task solvability before training.
envs/harbor_env/examples/quickstart.py Demonstrates a complete solve-and-evaluate episode against a running server.
envs/harbor_env/examples/tasks/fix-sum-bug/task.toml Bundled example Harbor task config used for end-to-end validation.
envs/harbor_env/examples/tasks/fix-sum-bug/instruction.md Example task instruction prompt served on reset.
envs/harbor_env/examples/tasks/fix-sum-bug/environment/stats.py Seeded “buggy” starting state for the example task.
envs/harbor_env/examples/tasks/fix-sum-bug/tests/test.sh Verifier entrypoint that runs the grader and writes reward artifacts.
envs/harbor_env/examples/tasks/fix-sum-bug/tests/grade.py Standard-library grader that computes partial credit and writes reward.json + reward.txt.
envs/harbor_env/examples/tasks/fix-sum-bug/solution/solve.sh Oracle script that installs the reference solution into the workdir.
envs/harbor_env/examples/tasks/fix-sum-bug/solution/stats.py Reference solution implementation used by the oracle.
docs/source/environments/harbor.md Adds official docs page for the new Harbor environment.
docs/source/environments.md Adds Harbor to the environments catalog page.
docs/source/_toctree.yml Adds Harbor docs page to the documentation toctree.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

envs/harbor_env/server/harbor_env_environment.py:261

  • _do_write() doesn’t validate that action.path is non-empty. Since the model default is "", this can resolve to the workdir and then attempt to write to a directory path, producing an opaque backend exception. Reject empty/whitespace-only paths up front to keep the action contract clear and errors stable across backends.
    def _do_write(
        self, action: HarborAction, sandbox: Sandbox, task: HarborTask
    ) -> HarborObservation:
        del task
        target = resolve_within(sandbox.paths.workdir, action.path)
        sandbox.write_text(target, action.content)
        return self._observe(
            "write", output=f"wrote {len(action.content)} bytes to {action.path}"
        )
  • Files reviewed: 26/27 changed files
  • Comments generated: 2
  • Review effort level: Low

Comment thread envs/harbor_env/server/sandbox.py
Comment thread envs/harbor_env/server/harbor_env_environment.py
Comment thread envs/harbor_env/server/sandbox.py
Comment thread envs/harbor_env/server/sandbox.py
…ion, empty paths

Cursor Bugbot and Copilot both reviewed huggingface#1018. Four concrete findings, all real:

1. reset() left the previous episode in state (Bugbot, medium)
   close() tears the sandbox down first, but _state was only replaced after
   catalog.get() and sandbox.start(), either of which can raise. A failed reset
   therefore reported a task_id, workdir and reward for an episode that was no
   longer running. Now drops to a clean HarborState before the fallible work.

2. DockerSandbox.upload_dir() archived nested files twice (Copilot)
   rglob("*") yields directories as well as files and tarfile.add() recurses by
   default, so every nested file was added once via its parent and again on its
   own iteration. Passes recursive=False; directory entries are still emitted,
   so empty directories survive.

3/4. read/write accepted an empty path (Copilot; write was filed low-confidence)
   HarborAction.path defaults to "", which resolve_within() happily resolved to
   the working directory itself. For read that produced a confusing
   "no such file: ". For write it was destructive: the target split into
   ("/", "workspace"), so the upload would have dropped a regular file over the
   whole checkout. Fixed in resolve_within() rather than at the two call sites,
   since that is the single chokepoint every agent-supplied path passes through
   — "", "   ", "." and "a/.." are all rejected now.

Also, from Copilot's Tier 2 flag that evaluate/solve share the action schema
with agent actions: AGENT_ACTIONS and CONTROL_ACTIONS were decorative — exported
but never referenced. HarborActionType is now composed from AgentActionType and
ControlActionType instead of restating the five strings, so a new action has to
be classified as one or the other to exist at all, and a test asserts the two
sets remain a partition of the handler table. That does not resolve the
boundary question itself, which is left for human review.

Tests: 8 added (58 total). Each regression test was confirmed to fail against
the unfixed code before being kept. Full suite 1471 passed, 130 skipped.
Copilot AI review requested due to automatic review settings July 28, 2026 15:11
@thegovind

Copy link
Copy Markdown
Collaborator Author

Thanks both — all four findings were real, and they're fixed in f568159. Each regression test was confirmed to fail against the unfixed code before I kept it, so none of these can quietly come back.

Tier 1

1. Failed reset left the previous episode in state (@cursor, medium) — correct, and the sequencing was the giveaway: close() tears the sandbox down first, but _state was only replaced after catalog.get() and sandbox.start(), either of which can raise. A failed reset therefore reported a task_id, workdir and reward for an episode that no longer existed. step() was safe (it raises SandboxError), but a trainer reading state would have been misled. Now drops to a clean HarborState immediately after close(), before anything fallible. The catalog listing deliberately survives, since it isn't episode state.

2. upload_dir() archived nested files twice (@copilot) — correct. rglob("*") yields directories as well as files and tarfile.add() recurses by default, so every nested file went in once through its parent and again on its own iteration. Now passes recursive=False; directory entries are still emitted so empty directories survive. It never surfaced because the bundled task and Repo2RLEnv's tasks both have flat tests/ dirs — the new test uses a nested one.

3/4. Empty path on read/write (@copilot) — correct, and I'd flag that the write case is more serious than its "low confidence" filing suggests. resolve_within() resolved "" to the working directory itself, and DockerSandbox.write_text() splits that into ("/", "workspace") — so an empty path would have dropped a regular file over the entire checkout. Fixed in resolve_within() rather than at the two call sites, since that's the single chokepoint every agent-supplied path passes through; "", " ", "." and "a/.." are all rejected now.

Tier 2 — alignment flags

evaluate/solve share the action schema with agent actions. Fair hit, and it exposed something worse than the design question: AGENT_ACTIONS and CONTROL_ACTIONS were decorative — exported but never referenced by anything. So the split was documentation that nothing enforced, and a new action type could have been added without anyone classifying it.

Partially addressed: HarborActionType is now composed from AgentActionType and ControlActionType rather than restating the five strings, so a new action has to be classified as one or the other to exist at all, and a test asserts the two sets stay a partition of the handler table.

That kills the drift risk but does not resolve the boundary question, so I've left it open for @Darktex. For context on where I think it actually stands: harbor_env exposes no MCP tool surface, so there is no channel by which an agent reaches step() directly — the trainer mediates every call, and the two sets document what belongs in a policy's action space. If that's considered insufficient, the honest fix is a server-side lockout (refuse control actions unless explicitly enabled), which I'd rather add on your call than guess at.

local mode executes as the server user. Agreed, and I don't want to soften it — it's documented in both the README and the LocalSandbox docstring as a filesystem boundary, not a security one. Worth an explicit accept/reject from a maintainer given it's the default mode. If the default should be docker with local opt-in, that's a one-line change and I'm happy to make it.


Full suite after the fixes and the main merge: 1564 passed, 131 skipped; 58 harbor tests (8 new). Also re-verified end to end against the updated core — server boot, WebSocket episode, and the three-runtime parity check (harbor run 0.20.0 / local / docker, all 0.600 → 1.000).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (3)

envs/harbor_env/server/harbor_env_environment.py:211

  • After evaluate ends the episode (done=True and state.evaluated=True), step() still allows further actions because _require_episode() continues to succeed. This contradicts the one-shot episode contract and can lead to trainers accidentally continuing to mutate a finished episode.
        self._state.step_count += 1
        self._state.last_action_type = action.action_type

        try:
            sandbox, task = self._require_episode()
            return self._handlers[action.action_type](action, sandbox, task)

envs/harbor_env/server/task.py:197

  • The needs_image docstring says any docker_image implies the starting state lives in an image, but the implementation treats tasks with environment/ seed files as not needing Docker even if docker_image is set. Please update the docstring so it matches the actual semantics (seed files => local mode allowed).
        A task that ships seed files carries its own starting state and can be
        reproduced anywhere. Anything else (a `Dockerfile`, a compose file, or a
        pre-built `docker_image`) puts the starting state in an image, so only
        the Docker execution mode can run it faithfully.

envs/harbor_env/models.py:79

  • HarborObservation.success is documented as being independent of command exit status ("a failing test command is a successful exec"), but the implementation/tests set success=False when exec exits non-zero. This docstring should be corrected to match the actual wire semantics so downstream consumers interpret success correctly.
        success (`bool`):
            Whether the action itself completed. Unrelated to the reward: a
            failing test command is a successful `exec`.
  • Files reviewed: 26/27 changed files
  • Comments generated: 0 new
  • Review effort level: Low

thegovind added a commit to thegovind/Repo2RLEnv that referenced this pull request Jul 28, 2026
…enEnv PR

Automated reviewers went over huggingface/OpenEnv#1018, which carries a
structurally parallel runtime. All four findings applied here too — this repo
just had no bot looking at it.

1. reset() left the previous episode in state
   close() tears the container down first, but _state was only replaced after
   tasks.get() and sandbox.start(), either of which can raise. A failed reset
   reported a task_id, workdir and reward for an episode that was gone. Now
   drops to a clean Repo2RLEnvState before the fallible work.

2. DockerSandbox.upload_dir() archived nested files twice
   rglob("*") yields directories as well as files and tarfile.add() recurses by
   default, so anything nested was added once via its parent and again on its
   own iteration. Passes recursive=False; directory entries are still emitted,
   so nested staging (e.g. the tests/verifier.py + tests/*.json layout our
   pipelines emit as aux_files) keeps working.

3. Degenerate paths resolved to the checkout itself
   Repo2RLEnvAction.path defaults to "", and resolve_within() resolved "", ".",
   "   " and "a/.." to the working directory. For read that meant catting a
   directory; for write it was destructive — the target split into
   ("/", "workspace"), so an empty path would have dropped a regular file over
   the entire checkout. Rejected in resolve_within(), the single chokepoint
   every agent-supplied path passes through.

4. AGENT_ACTIONS / CONTROL_ACTIONS were decorative
   Exported but never referenced, and the test asserting them merely restated
   the same literals, so it would have passed no matter what the server
   handled. ActionType is now composed from AgentActionType | ControlActionType
   rather than restating the five strings, and the test asserts the two sets are
   a partition of the environment's actual handler table.

Verification: each regression test was confirmed to fail against the unfixed
code before being kept. Also re-ran the Docker end-to-end with a task whose
tests/ has a nested subdirectory — the verifier reads tests/helpers/score.py,
proving recursive=False did not break nested staging — and confirmed an empty
write is now rejected with the checkout left intact.

Tests: 39 (6 added). Full suite 739 passed.
@burtenshaw

burtenshaw commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Three minor issues for completeness:

  • [p1] Local mode is the default and runs arbitrary shell commands with the full server environment; callers can read secrets/the oracle and escape the episode root. I'd feel more comfortable if we defaulted to docker.
  • [p1] Harbor network, resource, and user policies are ignored; e.g. no-network tasks still get Docker default networking.
  • [p2] the solve and evaluate split is not actually enforced. And evaluate returns done=True, but later actions still run and can return done=False.

…isode terminal

Addresses the three issues raised in review, plus two related holes found while
verifying them.

## [p1] local mode was the default and leaked too much

- `HARBOR_MODE` now defaults to `docker`. It is the faithful backend and the
  only one that can enforce a task's declared policies; `local` is an explicit
  opt-in for Docker-less hosts such as Spaces.
- `LocalSandbox.exec` no longer hands subprocesses the server's whole
  environment. Task commands are attacker-controlled input, so they now start
  from a short allowlist (PATH, HOME, LANG, …) and the process's API keys and
  Hub tokens stay out. A task that needs a secret declares it in
  `[environment].env`, which is resolved explicitly.

While confirming the escape, two further leaks turned up:

- `reset()` put the task's server-side directory in the observation, which
  reaches the agent. In local mode `exec cat <path>/solution/…` then read the
  oracle outright. `HarborTask.summary()` no longer carries `path`;
  `HarborState.task_path` still does, since state is infrastructure-side.
- `HARBOR_TESTS_DIR` / `HARBOR_SOLUTION_DIR` / `HARBOR_LOGS_DIR` were exported
  into agent commands. `SandboxPaths.as_env(agent_visible=True)` now withholds
  them; the verifier and oracle still receive the full set.
- `stage_tests` / `stage_solution` now wipe the destination first. The agent
  shares the filesystem and could pre-create `/tests`; a planted file survived
  staging and was read by the verifier. Regression test confirms it scored 1.0
  before this change.

## [p1] network, resource and user policies were ignored

`task.py` now parses `[environment].network_mode` / `allowed_hosts` / `cpus` /
`memory_mb` / `storage_mb` / `gpus` / `workdir` and `[agent].user` /
`[verifier].user`, including the deprecated `allow_internet` spelling.

- Docker: `no-network` becomes `network_mode="none"`, cpus/memory become real
  container limits, and each phase runs as its declared user. Verified against a
  live container — `getent hosts pypi.org` fails and `memory.max` reads
  536870912.
- Unenforceable policies are refused rather than approximated: `allowlist`
  (needs a filtering proxy) and `gpus`. An unknown `network_mode` is a parse
  error, so it can never silently degrade to `public`.
- Phases that disagree get the restrictive setting — a container has one
  network, and resolving that conflict permissively would hand a sandboxed
  verifier the internet.
- Local: refuses any task declaring a policy it cannot enforce, naming
  HARBOR_MODE=docker, instead of running it unconstrained and reporting a score
  the task never sanctioned.

## [p2] evaluate/solve split unenforced; done walked back

- After `evaluate` the episode is terminal: every later action is refused and
  keeps reporting `done=True`. Previously a post-evaluate `exec` returned
  `done=False`, contradicting a terminal state already reported to the trainer.
- `allow_control_actions` (env `HARBOR_ALLOW_CONTROL_ACTIONS=0`) makes the
  server refuse `evaluate`/`solve` outright, so the agent/orchestration split is
  enforced rather than left to the caller's wiring. Left on by default, since a
  run that cannot score itself is only useful for agent-facing deployments.

Tests: 14 added (72 total). Each regression test was confirmed to fail against
the unfixed code. Existing tests now pass `mode="local"` explicitly rather than
relying on the old default. Full suite 1578 passed, 131 skipped.
Copilot AI review requested due to automatic review settings July 29, 2026 13:40
Comment thread envs/harbor_env/server/harbor_env_environment.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (4)

envs/harbor_env/server/task.py:634

  • _positive_float() currently accepts bool values because bool is a subclass of int in Python (e.g. True becomes 1.0). This can silently treat malformed timeout_sec / policy values as valid.
def _positive_float(value: Any, fallback: float) -> float:
    try:
        parsed = float(value)  # type: ignore[arg-type]
    except (TypeError, ValueError):
        return fallback
    return parsed if parsed > 0 else fallback

envs/harbor_env/server/harbor_env_environment.py:103

  • Docstring says mode defaults to $HARBOR_MODE then "local", but the implementation uses DEFAULT_MODE="docker". This mismatch can mislead users about the default execution backend.
        mode (`str`, *optional*):
            Sandbox backend, `"local"` or `"docker"`. Defaults to `$HARBOR_MODE`,
            then `"local"`.

envs/harbor_env/server/task.py:533

  • resolve_task_source() treats any existing path as a valid task source, even if it is a file. This can lead to confusing downstream errors (empty catalog) instead of failing fast with an actionable message.
    local = Path(text).expanduser()
    if local.exists():
        return local.resolve()

envs/harbor_env/server/task.py:626

  • _positive_int() currently accepts bool values because bool is a subclass of int in Python (e.g. True becomes 1). This can silently mis-parse malformed task.toml policy values instead of rejecting them.

This issue also appears on line 629 of the same file.

def _positive_int(value: Any) -> int | None:
    try:
        parsed = int(value)  # type: ignore[arg-type]
    except (TypeError, ValueError):
        return None
    return parsed if parsed > 0 else None
  • Files reviewed: 26/27 changed files
  • Comments generated: 0 new
  • Review effort level: Low

thegovind added a commit to thegovind/Repo2RLEnv that referenced this pull request Jul 29, 2026
A maintainer reviewed huggingface/OpenEnv#1018, which carries a structurally
parallel runtime, and raised three issues. Two apply here in full; the third
(local-mode escapes) does not, since this runtime is Docker-only.

## Task policies were ignored

`dataset.py` now parses `[environment].network_mode` / `cpus` / `memory_mb` /
`gpus` / `workdir` and `[agent].user` / `[verifier].user`, including the
deprecated `allow_internet` spelling.

`_container_limits()` translates them into `containers.run` kwargs: no-network
becomes an isolated network, cpus/memory become real container limits, and each
phase runs as its declared user. What cannot be enforced faithfully is refused
rather than approximated — `allowlist` needs a filtering proxy and `gpus` needs
accelerators, so both point the caller at `harbor run`. An unrecognized
network_mode is a parse error, never a silent downgrade to `public`: quietly
granting a sandboxed task the internet is the exact failure this prevents.

Our pipelines do not emit these fields today, but the format allows them and a
task that declares one was being run unconstrained and scored anyway.

## The episode never terminated, and the action split was decorative

- After `evaluate`, `step()` refused nothing: a later `exec` returned
  `done=False`, contradicting a terminal state already reported to the caller.
  The episode is now terminal until `reset()`.
- `allow_control_actions` (env `REPO2RLENV_ALLOW_CONTROL_ACTIONS=0`) makes the
  server refuse `evaluate`/`solve`, so the agent/orchestration split is enforced
  rather than left to the caller's wiring.

## Two leaks found while verifying the above

- `HARBOR_TESTS_DIR` / `HARBOR_SOLUTION_DIR` / `HARBOR_LOGS_DIR` were exported
  into agent commands, handing a policy the location of the grading logic and
  the answer. `SandboxPaths.as_env(agent_visible=True)` withholds them; the
  verifier and oracle still receive the full set.
- `stage_tests` / `stage_solution` now wipe the destination first. The agent
  shares the container and can pre-create `/tests`; anything planted there
  previously survived staging and was read by the verifier.

Tests: 6 added (45 total). Full suite 740 passed. `tests/test_e2e_public.py::
test_e2e_public_trl` fails on a clean tree too — it queries live GitHub and
TRL's two most recent PRs are currently docs-only.
Self-review of the policy work found that honouring [agent].user broke every
task that declares one. The Harbor directories are created by the image's
default account, so an agent running as `nobody` inherited a working directory
it could not write to: `touch probe.txt` returned Permission denied, and every
`write` action would have failed. The same applied to [verifier].user and
/logs/verifier, which would have left those tasks unscorable.

Sandbox.chown() now hands the working directory and agent log dir to the agent
user at start, and the verifier log and tests dirs to the verifier user before
the verifier runs. Verified against a live container: the agent is `nobody`, the
workdir is writable, and the task still grades 0.6.

Also notes that storage_mb needs a quota-capable storage driver; where one is
absent Docker rejects the container, which is the fail-closed outcome.
Copilot AI review requested due to automatic review settings July 29, 2026 13:57
@thegovind

Copy link
Copy Markdown
Collaborator Author

Thanks Ben — all three were real, and hunting them turned up three more. Fixed across 592de2b2, 270a4579 (and f5681594 earlier).

[p1] local mode default and its blast radius

Agreed, and the escape was worse than described. reset() was putting the task's server-side directory in the observation, which reaches the agent — so in local mode:

obs = env.reset(task_id="fix-sum-bug")
env.step(HarborAction(action_type="exec",
                      command=f"cat {obs.info['task']['path']}/solution/stats.py"))
# → the oracle, in full

That is a complete reward-hacking hole, not just an isolation weakness. Changes:

  • HARBOR_MODE now defaults to docker. local is an explicit opt-in for Docker-less hosts.
  • summary() no longer carries path. HarborState.task_path still does — state is infrastructure-side.
  • exec no longer inherits the server's environment. It was passing all of os.environ, so a task command could read HF_TOKEN and any cloud credential the process held. It now starts from a short allowlist (PATH, HOME, LANG, …); a task needing a secret declares it in [environment].env, which is resolved explicitly.
  • HARBOR_TESTS_DIR / HARBOR_SOLUTION_DIR / HARBOR_LOGS_DIR are withheld from agent commands. The verifier and oracle still get the full set — they're the only ones who need to know where they are.
  • stage_tests/stage_solution wipe the destination first. The agent shares the filesystem and can pre-create /tests; I have a regression test where a planted file survived staging, was read by the verifier, and scored 1.0.

[p1] network / resource / user policies ignored

Correct — parsed and enforced now, against Harbor's own schema (NetworkMode, EnvironmentConfig, AgentConfig/VerifierConfig in harbor.models.task.config):

Declared Behaviour
network_mode = "no-network" container gets network_mode="none"
cpus, memory_mb, storage_mb real container limits
[agent].user / [verifier].user that phase runs as the named account
[environment].workdir overrides the image WORKDIR
network_mode = "allowlist", gpus refused — needs a filtering proxy / accelerators; points at harbor run
unknown network_mode parse error, never a silent downgrade to public

Verified against a live container, not just unit-tested: getent hosts pypi.org fails and /sys/fs/cgroup/memory.max reads 536870912 for memory_mb = 512.

Two judgement calls worth your eyes:

  • Phases that disagree (say a public agent and a no-network verifier) get the restrictive setting for the whole episode. A container has one network; resolving that conflict permissively would hand a sandboxed verifier the internet.
  • local mode now refuses any task declaring a policy it can't enforce, rather than running it unconstrained and reporting a score the task never sanctioned.

[p2] split unenforced, and done walked back

Both fixed:

  • The episode is terminal. After evaluate, every later action is refused and keeps done=True. You're right that it previously returned done=False after already reporting done=True — that's a straight contradiction to a trainer.
  • HARBOR_ALLOW_CONTROL_ACTIONS=0 makes the server refuse evaluate/solve outright, so the split is enforced rather than left to caller wiring. Default on, since a run that can't score itself is only useful for agent-facing deployments. Separately, HarborActionType is now composed from the agent and control literals rather than restating them, so a new action has to be classified to exist.

One bug I introduced and caught

Honouring [agent].user initially broke every task that declares one: the Harbor directories are created by the image's default account, so an agent running as nobody got a workdir it couldn't write to — touch returned Permission denied. Sandbox.chown() now hands the workdir and agent log dir to the agent user, and the verifier log/tests dirs to the verifier user. Re-verified on a live container: agent is nobody, workdir writable, task still grades 0.6.


Tests: 73 (15 added). Each regression test was confirmed to fail against the unfixed code before being kept. Existing tests now pass mode="local" explicitly instead of relying on the old default. Full suite 1579 passed, 131 skipped.

The same defects were present in the sibling Repo2RLEnv PR — same code shape, same author — so they're ported there too.

Two things I deliberately did not decide unilaterally:

  1. local mode is still a filesystem boundary, not a security one. It's now opt-in, secret-free and policy-refusing, but exec still runs as the server's user. If you'd rather it be gated behind an explicit HARBOR_ALLOW_LOCAL_MODE=1, say the word.
  2. storage_mb needs a quota-capable storage driver. Where one is absent Docker rejects the container — fail-closed, but the error is Docker's rather than ours. Happy to wrap it if you'd prefer a friendlier message.



@lru_cache(maxsize=None)
def resolve_task_source(source: str | os.PathLike[str] | None) -> Path:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PathLike task source crashes

Medium Severity

resolve_task_source is wrapped in lru_cache but accepts os.PathLike values such as pathlib.Path. Path objects are not hashable, so a normal call like resolve_task_source(Path("./tasks")) raises TypeError instead of resolving the directory, despite the documented API.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 270a457. Configure here.

Comment thread envs/harbor_env/server/harbor_env_environment.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (2)

envs/harbor_env/server/harbor_env_environment.py:103

  • The docstring says the mode default falls back to "local", but the implementation uses DEFAULT_MODE = "docker" when HARBOR_MODE is unset. This mismatch can mislead users reading the API docs and contradicts app.py/README which document docker as the default.
        mode (`str`, *optional*):
            Sandbox backend, `"local"` or `"docker"`. Defaults to `$HARBOR_MODE`,
            then `"local"`.

envs/harbor_env/server/task.py:533

  • resolve_task_source() treats any existing path as a valid task root, including regular files. If a file path is passed (by typo or env var), this will silently resolve and later yield an empty catalog instead of failing fast with an actionable error.
    local = Path(text).expanduser()
    if local.exists():
        return local.resolve()
  • Files reviewed: 26/27 changed files
  • Comments generated: 0 new
  • Review effort level: Low

Copilot AI review requested due to automatic review settings July 29, 2026 14:01
self.dockerfile is not None
or self.compose_file is not None
or bool(self.docker_image)
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local mode ignores docker_image

Medium Severity

When a task has environment/ seed files and a [environment].docker_image, needs_image returns false, so local mode runs on the host and never uses the declared image. Harbor still runs those seeds inside that image; local grading can fail or score incorrectly when the verifier or agent depends on packages only in the image.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 96bc040. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (2)

envs/harbor_env/models.py:80

  • HarborObservation.success is documented as being independent of command exit status, but the server sets success based on ExecResult.ok (exit code 0 and not timed out). The docstring should match the actual wire semantics so clients/tests interpret it consistently.
        success (`bool`):
            Whether the action itself completed. Unrelated to the reward: a
            failing test command is a successful `exec`.
        error (`str`):

envs/harbor_env/server/harbor_env_environment.py:103

  • Docstring says mode defaults to "local", but the implementation uses DEFAULT_MODE = "docker" when HARBOR_MODE is unset. This can mislead users about the default execution backend.
        mode (`str`, *optional*):
            Sandbox backend, `"local"` or `"docker"`. Defaults to `$HARBOR_MODE`,
            then `"local"`.
  • Files reviewed: 26/27 changed files
  • Comments generated: 0 new
  • Review effort level: Low

…ardening

A rubber-duck pass over 592de2b/270a4579 found that several of those fixes were
incomplete, and that one of them had silently defanged an existing test.

## The oracle was still reachable in local mode

Removing `path` from `HarborTask.summary()` was not enough: `HarborState`
carried `task_path`, and `state` rides the same socket as `step`. A caller that
could act could still read the host path and `cat <path>/solution/…`. The field
is gone.

## read/write bypassed [agent].user entirely

`read_text()` ran `cat` as the image's default account and `write_text()`
extracted a root-owned tar through the archive API, so a task declaring an
unprivileged agent still got root's reach through those two actions.
Demonstrated before the fix: a symlink in the working directory plus `read`
returned `/etc/shadow`, and plus `write` created a root-owned file under
`/root`. Both now run as the declared user, with writes going through the shell
so the kernel applies its permissions. Re-verified against a live container —
both attempts are refused and agent-written files are owned by `nobody`.

## Failed evaluation left the tests exposed and the episode running

`evaluated` was set only after staging, chowning and executing, so a failure in
between left `/tests` readable with `done=False`. The episode now terminates
before anything that can fail.

## storage_mb was not fail-closed

Docker accepts `storage_opt` on drivers that never enforce it — the review
demonstrated a 64 MiB write inside a 32 MiB "limit". A task that believes it is
capped and is not is worse than one told up front we cannot cap it, so this is
now refused like `allowlist` and `gpus`.

## Malformed limits read as "no limit"

`_positive_int` turned `0`, `-4` and `"lots"` into `None`, which meant both an
unconstrained container *and* no refusal from the local backend. Invalid values
are now `TaskFormatError`.

## Phase overrides did not replace the baseline

`NetworkPolicy.modes` unioned the baseline unconditionally, so a task whose
agent and verifier both declared `public` over a `no-network` baseline had its
network disabled anyway. Only the effective per-phase modes count now.

## An existing security test had gone vacuous

`test_planted_reward_files_are_discarded` planted through `$HARBOR_LOGS_DIR`,
which 592de2b hid from agent commands — so the plant silently failed and the
test passed for the wrong reason. It now uses the real path and asserts the
plant landed before checking it is discarded.

## Smaller

- Agent-supplied `timeout_s` is capped at the configured ceiling instead of
  overriding it; it arrives on the wire and was unbounded.
- `validate_taskset.py` passes `allow_control_actions=True` — it *is* the
  orchestration, and was inheriting the deployment's flag.
- The README no longer claims local mode keeps secrets from tasks. It keeps them
  out of the environment, but `/proc/<ppid>/environ` still exposes them; that is
  hygiene, not containment, and is now described as such.

Tests: 78 (5 added). Full suite 1584 passed, 131 skipped.
Copilot AI review requested due to automatic review settings July 29, 2026 14:30
@thegovind

Copy link
Copy Markdown
Collaborator Author

Follow-up: I ran an independent review pass over my own fixes above, and several of them were incomplete. Fixed in dd2232b7.

Worth reading if you're reviewing the security claims, because one of my fixes silently defanged an existing test.

The oracle was still reachable in local mode

Removing path from summary() wasn't enough — HarborState.task_path carried the same host path, and state rides the same socket as step. Anyone who could act could still read it and cat <path>/solution/…. Field removed.

read/write bypassed [agent].user completely

This is the one I'm most glad turned up. read_text() ran cat as the image default (root) and write_text() extracted a root-owned tar through the archive API — so the user policy only ever applied to exec. Demonstrated before the fix, with [agent].user = "nobody":

ln -sf /etc/shadow leak   →  read("leak")            → returned /etc/shadow
ln -sfn /root rootdir     →  write("rootdir/x")      → created a root-owned file in /root

Both now run as the declared user; writes go through the shell so the kernel applies permissions rather than tar extraction. Re-verified live — both attempts refused, agent-written files owned by nobody, task still grades 0.6.

storage_mb was not fail-closed — my earlier comment was wrong

I claimed Docker rejects storage_opt on drivers that can't honour it. It doesn't: the review demonstrated a 64 MiB write inside a 32 MiB "limit", with the option sitting in HostConfig doing nothing. A task that believes it's capped and isn't is worse than one told up front, so it's now refused alongside allowlist and gpus.

Malformed limits read as "no limit"

cpus = 0 / -4 / "lots" all became None → unconstrained container and no refusal from local mode. Now a TaskFormatError.

Phase overrides didn't replace the baseline

NetworkPolicy.modes unioned the baseline unconditionally, so a task whose agent and verifier both declared public over a no-network baseline had its network disabled anyway. Only effective per-phase modes count now.

An existing test had gone vacuous

test_planted_reward_files_are_discarded planted via $HARBOR_LOGS_DIR — which my own agent_visible change had just hidden from agent commands. So the plant silently failed and the test passed for the wrong reason. It now uses the real path and asserts the plant landed before checking it's discarded.

Smaller

  • Agent-supplied timeout_s is capped at the configured ceiling rather than overriding it (it arrives on the wire, and was unbounded).
  • validate_taskset.py now passes allow_control_actions=True — it is the orchestration and was inheriting the deployment's flag, which would have broken it in a locked-down config. Good catch on that one.
  • Corrected a claim in the README. I'd said local mode keeps secrets from tasks. It keeps them out of the environment, but /proc/<ppid>/environ still exposes them. That's hygiene, not containment, and the docs now say so.

78 tests (20 added overall), full suite 1584 passed, 131 skipped. Ported to the Repo2RLEnv PR too.

Two things I've left for you rather than deciding unilaterally, both of which the review flagged as architectural:

  1. solve/evaluate still share the wire schema with agent actions. HARBOR_ALLOW_CONTROL_ACTIONS=0 is honestly a kill switch, not a privilege split — it disables scoring too. A real split needs either separate agent/control wire models or an authenticated control channel, which is an OpenEnv-wide decision rather than something this env should invent. Happy to implement whichever shape you and @Darktex prefer.
  2. local mode remains a filesystem boundary only. It's now opt-in, secret-light and policy-refusing, but exec still runs as the server's user in a shared process namespace. If you'd rather it be gated behind an explicit HARBOR_ALLOW_LOCAL_MODE=1, or dropped from the shipped image entirely, say the word.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (3)

envs/harbor_env/server/sandbox.py:646

  • DockerSandbox.write_text(): when user is set, the implementation writes via shell redirection but does not create parent directories first (unlike the root/put_archive path and LocalSandbox). Writing to a nested path like subdir/file.txt will fail even though Sandbox.write_text() promises to auto-create parents.
    def write_text(self, path: str, content: str, user: str | None = None) -> None:
        if user:
            # put_archive extracts as root, which would let an unprivileged
            # agent create root-owned files — and, through a symlink out of the
            # working directory, create them anywhere. Write through the shell

envs/harbor_env/server/harbor_env_environment.py:103

  • Docstring mismatch: the constructor defaults mode to $HARBOR_MODE and then DEFAULT_MODE (which is "docker"), but the docstring says it falls back to "local". This is user-facing API documentation for the environment.
        mode (`str`, *optional*):
            Sandbox backend, `"local"` or `"docker"`. Defaults to `$HARBOR_MODE`,
            then `"local"`.

envs/harbor_env/server/task.py:397

  • HarborTask.summary() docstring references HarborState.task_path, but HarborState does not define such a field (and tests assert task directory paths are not leaked via state). This comment is misleading for maintainers.
        Deliberately omits [`path`]: the observation reaches the agent, and
        naming the task directory on the server would point straight at
        `solution/` and `tests/`. [`HarborState.task_path`] still carries it for
        training orchestration, which is on the infrastructure side.
  • Files reviewed: 26/27 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment on lines +851 to +853
if not target.startswith(base_norm.rstrip("/") + "/"):
raise ValueError(f"path escapes the working directory: {relative!r}")
return target
Copilot AI review requested due to automatic review settings July 29, 2026 14:38
except (TypeError, ValueError) as exc:
raise TaskFormatError(
f"[environment].{field} must be a positive integer, got {value!r}"
) from exc

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TOML floats break resource limits

Low Severity

_positive_int parses resource fields with int(str(value)), so legitimate TOML numeric values like cpus = 2.0 raise TaskFormatError instead of loading the task, even when the value is a whole number.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 99f473b. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (3)

envs/harbor_env/server/sandbox.py:650

  • In DockerSandbox.write_text, the user branch writes via shell redirection but never ensures the parent directory exists. This violates the method contract (“creating parent directories as needed”) and will fail for writes to nested paths (e.g. path="/app/src/foo.py"). Create the parent directory before executing the write command (same as the put_archive branch).
    def write_text(self, path: str, content: str, user: str | None = None) -> None:
        if user:
            # put_archive extracts as root, which would let an unprivileged
            # agent create root-owned files — and, through a symlink out of the
            # working directory, create them anywhere. Write through the shell

envs/harbor_env/server/harbor_env_environment.py:103

  • The mode parameter docstring says the default falls back to "local", but the implementation uses DEFAULT_MODE = "docker" when HARBOR_MODE is unset. This is user-facing documentation and should match the actual default behavior.
        mode (`str`, *optional*):
            Sandbox backend, `"local"` or `"docker"`. Defaults to `$HARBOR_MODE`,
            then `"local"`.

envs/harbor_env/server/Dockerfile:52

  • This comment says timeout bounds commands in the local backend, but the local backend uses Python’s subprocess.run(..., timeout=...). The timeout binary is actually used by the Docker backend (DockerSandbox._with_timeout) inside the task container. Updating the comment will avoid misleading operators about where command bounding happens.
# Harbor verifiers commonly shell out to git, bash and coreutils; `timeout`
# bounds commands in the local backend's subprocesses, and curl backs HEALTHCHECK.
RUN apt-get update && \
  • Files reviewed: 26/27 changed files
  • Comments generated: 0 new
  • Review effort level: Low

Cursor and Copilot both flagged the same hole independently, and they were
right: `resolve_within()` is a purely lexical check, so it cannot see a symlink
the agent planted with `exec`. Demonstrated before the fix:

    exec  ln -s <tests dir> link
    read  path="link/grade.py"   -> returned the grader's source

That defeats the documented guarantee that `/tests` and `/solution` are
unreachable through `read`/`write` — the agent could read the grading logic, and
`write` could plant files the verifier would then execute.

Agent paths now go through `Sandbox.resolve_agent_path()`, which keeps the
lexical check and then canonicalizes on the sandbox's own filesystem:
`Path.resolve()` locally and `readlink -m` in the container. `-m` rather than
`-f` because a `write` may legitimately create directories that do not exist
yet, while symlinks that *do* exist are still resolved — which is the only part
confinement depends on.

The base is canonicalized too. Without that the fix rejected everything on
macOS, where the episode root lives under `/var`, itself a link to
`/private/var`. Both cases are covered by tests.

Also, from the same round of review:

- Local mode now flags a task that declares `[environment].docker_image` while
  running on the host toolchain. Seed files make the *state* reproducible, but
  the image supplies the toolchain, and a version skew would otherwise surface
  as a mis-graded episode. Warned in the log and surfaced in the reset
  observation as `info["toolchain_warning"]`.

Two other items from that round needed no change:

- "Failed evaluate leaves the episode open" was reported against 270a457 and
  was already fixed in dd2232b, which terminates before the verifier runs.
  Verified: a task with no `tests/test.sh` returns `done=True`, as does every
  action after it.
- "PathLike task source crashes" does not reproduce. `pathlib.Path` is hashable,
  so `lru_cache` accepts it; `resolve_task_source(Path("..."))` resolves
  normally.

Tests: 81 (3 added). The symlink test was confirmed to fail against the
pre-fix lexical check. Full suite 1587 passed, 131 skipped.
Copilot AI review requested due to automatic review settings July 29, 2026 14:49
@thegovind

Copy link
Copy Markdown
Collaborator Author

Thanks both — the symlink finding was the real one, and it was a good catch. Fixed in 5079a775 (rebased onto the latest main merge).

Symlink escape from read/write — confirmed and fixed

@Copilot and @cursor flagged this independently and were right: resolve_within() is purely lexical, so it can't see a link the agent planted with exec. Reproduced before the fix:

exec  ln -s <tests dir> link
read  path="link/grade.py"   →  returned the grader's source

That defeats the documented guarantee that /tests and /solution are unreachable — the agent could read the grading logic, and write could plant files the verifier then executes.

Agent paths now go through Sandbox.resolve_agent_path(), which keeps the lexical check and then canonicalizes on the sandbox's own filesystemPath.resolve() locally, readlink -m in the container.

Two details worth flagging, since both bit me:

  • -m not -f. readlink -f requires every parent to exist, which broke legitimate nested writes (write path="pkg/mod/new.py"). -m still resolves symlinks that do exist, which is the only part confinement depends on.
  • The base needs canonicalizing too. My first attempt rejected everything on macOS, where the episode root is under /var — itself a link to /private/var. Comparing a resolved path against an unresolved base fails closed on every access. Both cases now have tests.

Local mode ignoring docker_image — fair, now surfaced

@cursor is right that a task with seed files and [environment].docker_image runs on the host toolchain in local mode. The seed files make the state reproducible, but the image supplies the toolchain, and a skew would surface as a mis-graded episode rather than an error.

I didn't make this refuse outright — that would reject most useful tasks in local mode, including the bundled example — but it's no longer silent: a warning in the log, and info["toolchain_warning"] in the reset observation so a training run can see it. If you'd rather it hard-fail, that's a one-line change.

Two that needed no change

  • "Failed evaluate leaves episode open" — reported against 270a4579, already fixed in dd2232b7, which terminates before the verifier runs precisely so a failure mid-evaluate can't leave the episode live. Verified: a task with no tests/test.sh returns done=True, as does every action after it.
  • "PathLike task source crashes" — doesn't reproduce. pathlib.Path is hashable, so lru_cache accepts it fine:
    >>> resolve_task_source(Path("envs/harbor_env/examples/tasks"))
    PosixPath('/…/envs/harbor_env/examples/tasks')

81 tests (3 added). The symlink test was confirmed to fail against the pre-fix lexical check before I kept it. Full suite 1599 passed, 131 skipped after rebasing onto the latest main. Ported to the Repo2RLEnv PR as well.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 4 total unresolved issues (including 3 from previous reviews).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5079a77. Configure here.

raise ValueError(
f"path escapes the working directory through a link: {relative!r}"
)
return real

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Windows workdir confinement broken

Medium Severity

resolve_agent_path decides confinement with real.startswith(base.rstrip("/") + "/"), which assumes POSIX trailing slashes. On Windows, real_path returns backslash paths, so legitimate files under the workdir often fail the prefix test and read/write incorrectly raise path-escape errors—or checks behave inconsistently. resolve_within uses the same slash logic on Windows workdir strings.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 5079a77. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Comments suppressed due to low confidence (4)

envs/harbor_env/server/sandbox.py:717

  • When writing as a declared user, DockerSandbox.write_text() redirects into the target path without creating parent directories. This breaks writes to nested paths (which the Sandbox.write_text contract allows) for tasks that set [agent].user / [verifier].user.
            payload = base64.b64encode(content.encode("utf-8")).decode("ascii")
            result = self.exec(
                f"printf %s {_quote(payload)} | base64 -d > {_quote(path)}",
                timeout_s=120.0,
                user=user,

envs/harbor_env/server/harbor_env_environment.py:101

  • Docstring says the default sandbox mode falls back to "local", but the implementation uses DEFAULT_MODE = "docker" when HARBOR_MODE is unset. This can mislead users reading the API docs.
        mode (`str`, *optional*):
            Sandbox backend, `"local"` or `"docker"`. Defaults to `$HARBOR_MODE`,
            then `"local"`.

envs/harbor_env/models.py:80

  • HarborObservation.success is documented as "Unrelated to the reward: a failing test command is a successful exec", but the server sets success based on ExecResult.ok (exit_code==0 and not timed out). The docstring should reflect the actual semantics so clients don't misinterpret failures.
        success (`bool`):
            Whether the action itself completed. Unrelated to the reward: a
            failing test command is a successful `exec`.
        error (`str`):

envs/harbor_env/server/sandbox.py:698

  • DockerSandbox.read_text() currently returns None for any non-zero cat exit code (including permission errors or other failures). Callers treat None as "file missing" and will surface misleading FileNotFoundError messages; reward reading also loses useful diagnostics. Distinguish "missing" from other errors and raise on non-missing failures.
    def read_text(self, path: str, user: str | None = None) -> str | None:
        result = self.exec(f"cat {_quote(path)}", timeout_s=60.0, user=user)
        return result.output if result.ok else None
  • Files reviewed: 26/27 changed files
  • Comments generated: 0 new
  • Review effort level: Low

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants